The Courtroom-Observation Program of the Utah Judicial Performance Evaluation Commission

No. 1539825 (Dec. 18, 2009), http://papers.ssrn.com/ sol3/papers.cfm?abstract_id=1539825. 11. Seth S. Andersen, Judicial Retention Evaluation Programs, 34 LOY. L.A. L. REV. 1375 (2001). The State of Utah has recently introduced several innovations to its judicial performance evaluation program. Since the 1960s, such programs have been used in a growing number of states to inform decisions about judicial retention and to provide feedback to judges about their performance. The evaluations have traditionally been limited to surveys completed by a variety of stakeholders in the court system. But several states have begun to expand the scope of their evaluation programs beyond surveys, and one of these innovations is to observe judges in their courtrooms. Utah has developed its courtroom-observation program to become a major component of its overall judicial performance evaluation. This article describes the history of judicial performance evaluations in Utah, the introduction of courtroom observation by lay observers, and the recent innovations to the observation program, including its focus on procedural fairness, use of qualitative evaluation methods, and use of systematic content analysis of the observers’ narrative reports. BACKGROUND TO THE UTAH COURTROOM-OBSERVATION PROGRAM History of Judicial Performance Evaluation Judicial performance evaluation (JPE) has steadily grown in popularity since the first program in Alaska in 1967.1 In 1985, the American Bar Association issued its first guidelines for JPE, and by 1998 “approximately 23 states had some sort of JPE program in place or under development.”2 In 2005, the American Bar Association updated its JPE guidelines, proposing that all court systems implement a formal program to promote judicial self-improvement, enhance the quality of the judiciary, and provide relevant information to voters. These guidelines provide evaluative criteria in five areas (legal ability, integrity and impartiality, communication skills, professionalism and temperament, and administrative capacity), and suggest various aspects of evaluation design, including evaluating actual behaviors rather than the general qualities of judges, collecting data from multiple sources, and using experts for developing evaluation methods and for collecting and analyzing data.3 It is not clear that these guidelines have uniformly been followed, and David Brody has proposed a nationwide assessment of the processes currently in use in JPE programs.4 One issue seems clear from the JPE literature: reliance on surveys that result in a single metric of performance does not promote acceptance and trust of JPE by judges.5 While some studies of JPE programs find that that judges have predominantly positive comments about JPE, including the usefulness of feedback and the appropriateness of criteria,6 other studies document judicial mistrust of survey-based JPE. In a Colorado study, while judges reported positive comments about JPE, many criticized the survey methodology used as “unscientific” and “inherently flawed.” 7 Another study found that some JPE surveys appear to be “a fishing expedition trolling about for evidence of judicial misbehavior or performance problems”8 rather than an evaluation of clear performance standards. Analyses of survey data show that respondents tend to evaluate female and minority judges more harshly9 and that aggregating scores across a wide variety of criteria, or using a single measure for different kinds of courts, reduces the validity of the survey results.10 Concerns with survey methodologies provide one explanation for the slow adoption of JPE.11 The Courtroom-Observation Program of the Utah Judicial Performance Evaluation Commission Nicholas H. Woolf & Jennifer MJ Yim 84 Court Review Volume 47 12. Rebecca Love Kourlis & Jordan M. Singer, Using Judicial Performance Evaluations to Promote Judicial Accountability, 90 JUDICATURE 200 (2007). 13. For the most recent report of the independent citizen organization, Alaska Judicial Observers, Inc., on its in-court observations of Alaska judges, see Alaska Judicial Observers, 2010 Biennial Report, http://www.ajc.state.ak.us/retention/retent2010/judobse10.pdf. 14. The Fund for Modern Courts in New York has been training volunteers to observe judges and publicly report their observations since 1975. Recent reports may be found at http://www.modern courts.org/Programs/monitoring.html. A group in Minnesota, called WATCH, indicates on its website that it has trained volunteers to observe court hearings in Hennepin County, Minnesota. See http://www.watchmn.org/. 15. 2008 Utah Laws ch. 248, § 5, codified at Utah Code Ann. 78A-12-